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I. Real Party in Interest 
The assignee of the present application is Hewlett-Packard Development Company, 

L.P. 



200315391-1 Group Art Unit: 2628 

Serial No.: 10/684,030 



II. Related Appeals and Interferences 
There are no related appeals or interferences known to the Appellants. 
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III. Status of Claims 

Claims 1-33 are pending. Claims 1-33 are rejected. This Appeal involves Claims 1- 

33. 
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IV. Status of Amendments 
All proposed amendments have been entered. An amendment subsequent to the Final 
Action has not been filed. 
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V. Summary of Claimed Subject Matter 
Independent Claims 1, 13,21, and 33 of the present application pertain to various 
embodiments of culling view dependent visual data streams for a virtual environment. 

Independent Claim 1 recites, "A method for culling view dependent visual data 
streams for a virtual environment 300, 500." This embodiment is described at least at page 7 
line 5 to page 7 line 27; page 9 line 24 to page 10 line 20; Figure IB; and Figure 2. 
"Determining 210 a view volume 321 of a viewing participant 320, 520 within said virtual 
environment 300, 500, wherein said view volume 321 defines a field-of-view of said viewing 
participant 320, 520 within said virtual environment 300, 500" is described at least at step 
210, Figure 2; page 10 lines 4-9; page 1 1 line 16 to page 12 line 23; page 16 line 14 to page 
17 line 27; Figure 3; and Figure 5. "Determining 220 a proximity of a representation of an 
observed object 310, 510 in said virtual environment 300, 500 to said view volume 321" is 
described at least at step 220, Figure 2; page 10 lines 11-14; page 1 1 line 16 to page 12 line 
23; page 16 line 14 to page 17 line 27; Figure 3; and Figure 5. "Processing 230 a view 
dependent visual data stream of said observed object 310, 510 only when said representation 
is within a specified proximity 530 to said view volume 321" is described at least at step 230, 
Figure 2; page 10 lines 11-14; page 1 1 line 16 to page 12 line 23; page 16 line 14 to page 17 
line 27; page 14 line 14 to page 16 line 12; Figure 3; and Figure 5. 

Independent Claim 13 recites, "A system for culling view dependent visual data for a 
virtual environment 300, 500." This embodiment is described at least at page 18; Figure 6; 
page 7 line 5 to page 7 line 27; page 9 line 24 to page 10 line 20; Figure IB; and Figure 2. 
"A view volume generator 610 for determining 210 a view volume 321 of a viewing 
participant 320, 520 within said virtual environment 300, 500, wherein said view volume 321 
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defines a field-of-view of said viewing participant 320, 520 within said virtual environment 
300, 500" is described at least at page 18; page 10 lines 4-9; step 210, Figure 2; page 1 1 line 
16 to page 12 line 23; page 16 line 14 to page 17 line 27; Figure 3; and Figure 5. "A 
comparator 620 for determining 220 a proximity of a representation of an observed object 
310, 510 in said virtual environment 300, 500 to said view volume 321" is described at least 
at page 18; step 220, Figure 2; page 10 lines 11-14; page 1 1 line 16 to page 12 line 23; page 
16 line 14 to page 17 line 27; Figure 3; and Figure 5. "A processor 630 for processing 230 a 
view dependent visual data stream of said observed object 3 10, 5 10 only when said 
representation is within a specified proximity 530 to said view volume 321" is described at 
least at page 18; step 230, Figure 2; page 10 lines 1 1-14; page 1 1 line 16 to page 12 line 23; 
page 16 line 14 to page 17 line 27; page 14 line 14 to page 16 line 12; Figure 3; and Figure 5. 

Independent Claim 21 recites, "A computer system." This embodiment is described 
at least at page 5 lines 17-25; page 7 line 5 to page 7 line 27; page 9 line 24 to page 10 line 
20; Figure IB; and Figure 2. "A processor," is described at least at page 5 lines 17-25. "A 
computer readable memory coupled to said processor and containing program instructions 
that, when executed, implement a method for culling view dependent visual data streams for 
a virtual environment 300, 500," is described at least at page 5 line 17 to page 6 line 4; page 
7 line 5 to page 7 line 27; page 9 line 24 to page 10 line 20; Figure IB; and Figure 2. 
"Determining 210 a view volume 321 of a viewing participant 320, 520 within said virtual 
environment 300, 500, wherein said view volume 321 defines a field-of-view of said viewing 
participant 320, 520 within said virtual environment 300, 500" is described at least at page 10 
lines 4-9; step 210, Figure 2; page 1 1 line 16 to page 12 line 23; page 16 line 14 to page 17 
line 27; Figure 3; and Figure 5. "Determining 220 a proximity of a representation of an 
observed object 310, 510 in said virtual environment 300, 500 to said view volume 321" is 
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described at least at step 220, Figure 2; page 10 lines 11-14; page 1 1 line 16 to page 12 line 
23; page 16 line 14 to page 17 line 27; Figure 3; and Figure 5. "Processing 230 a view 
dependent visual data stream of said observed object 310, 510 only when said representation 
is within a specified proximity 530 to said view volume 321" is described at least at step 230, 
Figure 2; page 10 lines 11-14; page 1 1 line 16 to page 12 line 23; page 16 line 14 to page 17 
line 27; page 14 line 14 to page 16 line 12; Figure 3; and Figure 5. 

Independent Claim 33 recites, "A computer readable medium containing executable 
instructions which, when executed in a processing system, causes the system to perform the 
steps for a method of culling view dependent visual data streams for a virtual environment 
300, 500." This embodiment is described at least at page 5 line 17 to page 6 line 4; page 7 
line 5 to page 7 line 27; page 9 line 24 to page 10 line 20; Figure IB; and Figure 2. 
"Determining 210 a view volume 321 of a viewing participant 320, 520 within said virtual 
environment 300, 500, wherein said view volume 321 defines a field-of-view of said viewing 
participant 320, 520 within said virtual environment 300, 500" is described at least at page 10 
lines 4-9; step 210, Figure 2; page 1 1 line 16 to page 12 line 23; page 16 line 14 to page 17 
line 27; Figure 3; and Figure 5. "Determining 220 a proximity of a representation of an 
observed object 310, 510 in said virtual environment 300, 500 to said view volume 321" is 
described at least at step 220, Figure 2; page 10 lines 11-14; page 1 1 line 16 to page 12 line 
23; page 16 line 14 to page 17 line 27; Figure 3; and Figure 5. "Processing 230 a view 
dependent visual data stream of said observed object 310, 510 only when said representation 
is within a specified proximity 530 to said view volume 321" is described at least at step 230, 
Figure 2; page 10 lines 11-14; page 1 1 line 16 to page 12 line 23; page 16 line 14 to page 17 
line 27; page 14 line 14 to page 16 line 12; Figure 3; and Figure 5. 
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VI. Grounds of Rejection to Be Reviewed on Appeal 

1. Claims 1-4, 7-15, 17-24 and 27-33 are rejected under 35 U.S.C. §102(b) as being 
anticipated by U.S. Patent No. 5,745,126 by Jain et al (referred to hereinafter as 
"Jain"). 

2. Claims 5, 6, 16, 25 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Jain in view of U.S. Patent No. 6,100,862 by Sullivan (referred to hereinafter as 
"Sullivan"). 
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VII. Argument 

1. Whether Claims 14,5, 7-9, 1 1. 12. 14, 15, 18, and 19 are Anticipated Under 35 U.S.C. 
$ 102(e) by Jain. 

Appellants have reviewed the cited art and respectfully submits that the embodiments as 
recited in Claims 1, 4, 5, 7-9, 11, 12, 14, 15, 18, and 19 are not anticipated by Jain in view of the 
following rationale. 



MPEP §2131 provides: 

"A claim is anticipated only if each and every element as set forth in the claim is 
found, either expressly or inherently described, in a single prior art reference." 
Verdegaal Bros. v. Union Oil Co. of California, 814 F.2d 628, 631, 2 USPQ2d 1051, 
1053 (Fed. Cir. 1987). . . . "The identical invention must be shown in as complete 
detail as is contained in the ... claim." Richardson v. Suzuki Motor Co., 868 F.2d 
1226, 1236, 9 USPQ2d 1913, 1920 (Fed. Cir. 1989). The elements must be arranged 
as required by the claim. 

Claim 1 recites, 

A method for culling view dependent visual data streams for a virtual environment, 
comprising: 

determining a view volume of a viewing participant within said virtual 
environment, wherein said view volume defines a field-of-view of said viewing 
participant within said virtual environment; 

determining a proximity of a representation of an observed object in said 
virtual environment to said view volume; and 

processing a view dependent visual data stream of said observed object only 
when said representation is within a specified proximity to said view volume. 



A . Cited Art does not show the "Identical Invention" 

Appellants understand Jain to teach a method or system that enables each viewer of a 
broadcast event, such as a football game, to be their own "director" (col. 2 line 67) by 
selecting, for example, a particular object that appears in a scene or an event depicted in the 
scene that they are interested in (Col. 7 lines 36-42). In a specific example, a viewer may 
select that they are interested in a particular player or the football itself (Col. 7 lines 33-34). 
In this case, the selected object (foot ball or particular player) may be classified, tagged and 
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tracked (Col. 7 lines 51-64) to enable the viewer to view the selected object/event. It is 
possible for one of Jain's viewers to select a perspective that is not available from any 
camera (Col. 19 lines 5-6). Therefore, Appellants understand Jain, in one embodiment, to 
teach a view volume that surrounds a selected object or event. Since Jain's view volume 
surrounds a selected object or event, Appellants do not understand Jain's selected object or 
event to be capable of moving in or out of Jain's view volume for Jain's selected 
object/event. 

Appellants do not understand Jain's object bounding box to teach or suggest the 
embodiment recited by Claim 1 either. For example, Appellants understand Jain's object 
bounding box to be used as a bounding box for an object and therefore the object that the 
bounding box surrounds would not move in or out of its bounding box. Further, even when 
objects are within relative proximity of each other, they are still classified as separate objects. 
This is possible because of Jain's history and tracking mechanism (Col. 35 lines 35-51). 

Appellants do not understand Jain's cameras to teach or suggest the embodiment 
recited by Claim 1 either. For example, Appellants do not understand Jain to teach or 
suggest, among other things, processing a view dependent visual data stream of an observed 
object (observed by Jain's camera) only when said representation is within a specified 
proximity of a view volume of Jain's camera. 

Thus, Appellants do not understand Jain to teach or suggest " processing a view 
dependent visual data stream of said observed object only when said representation is within 
a specified proximity of said view volume" (emphasis added) where "said view volume 
defines a field-of-view of said viewing participant within said virtual environment" 
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(emphasis added) and said representation is "a representation of an observed object in said 
virtual environment" nor to teach or suggest "determining a proximity of a representation of 
an observed object in said virtual environment to said view volume," as recited by Claim 1. 

The Response to Arguments section states, "it is noted that the respective claim 
language is silent as to any limitations directed toward how said participant is graphically 
represented in said virtual environment." Appellants respectfully submit that Claim 1 recites, 
"wherein said view volume defines a field-of-view of said viewing participant . . .determining 
a proximity of a representation of an observed object in said virtual environment to said view 
volume . . .processing a view dependent visual data stream of said observed object only when 
said representation is within a specified proximity to said view volume" (emphasis added). 

For at least the reasons provided herein, Appellants submit that independent Claim 1 is 
not anticipated by Jain as the Rejection fails to establish a prima facie case for anticipation of 
Claim 1. For similar reasons, Appellants submit that independent Claims 13, 21 and 33 are not 
anticipated by Jain as the Rejection fails to establish a prima facie case for anticipation of 
Claims 13, 21 and 33. As such, Appellants submit that independent Claims 1, 13, 21 and 33 are 
in condition for allowance. Claims 1-12 depend from Claim 1. Claims 14-20 depend on Claim 
13. Claims 22-32 depend on Claim 21 . Hence, it is respectfully submitted that these dependent 
Claims are patentable over Jain for the reasons discussed above, and are in condition for 
allowance by virtue of their dependence upon a respective allowable base claim. 
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2. Whether Claims 5, 6, 16, 25 and 26 are Unpatentable Under 35 U.S.C. § 102(e) over Jain 
in view of Sullivan. 

Appellants have reviewed the cited art and respectfully submit that the embodiments as 
recited in Claims 5,6, 16, 25 and 26 are neither taught nor suggested by Jain or Sullivan, alone 
or in combination, in view of the following rationale. 



"As reiterated by the Supreme Court in KSR, the framework for the objective analysis 
for determining obviousness under 35 U.S.C. 103 is stated in Graham v. John Deere Co., 383 
U.S. 1, 148 USPQ 459 (1966). Obviousness is a question of law based on underlying factual 
inquiries" including "[ascertaining the differences between the claimed invention and the prior 
art" (MPEP 2141(11)). "In determining the differences between the prior art and the claims, the 
question under 35 U.S.C. 103 is not whether the differences themselves w ould have been 
obvious, but whether the claimed invention as a whole would have been obvious" (emphasis in 
original; MPEP 2141 .02(1)). Appellant notes that "[t]he prior art reference (or references when 
combined) need not teach or suggest all the claim limitations, however, Office personnel must 
explain why the difference(s) between the prior art and the claimed invention would have been 
obvious to one of ordinary skill in the art " (emphasis added; MPEP 2141(111)). 



Claim 1 recites, 

A method for culling view dependent visual data streams for a virtual environment, 
comprising: 

determining a view volume of a viewing participant within said virtual 
environment, wherein said view volume defines a field-of-view of said viewing 
participant within said virtual environment; 

determining a proximity of a representation of an observed object in said 
virtual environment to said view volume; and 

processing a view dependent visual data stream of said observed object only 
when said representation is within a specified proximity to said view volume. 
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As already discussed herein, Appellants respectfully submit that independent Claim 1 is 
patentable over Jain. Further, independent Claims 13 and 21 are patentable over Jain for similar 
reasons that Claim 1 is patentable over Jain. Appellants respectfully agree with the Office 
Action that Sullivan does not teach the embodiments recited by independent Claims 1,13 and 
21 . Claims 5 and 6 depend on independent Claim 1 . Claims 16 and 25 depend on independent 
Claim 13. Claim 26 depends on independent Claim 21. These dependent claims should be 
patentable for at least the reasons that their respective independent claims should be patentable. 
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Conclusion 

Appellants believe that pending Claims 1-4, 7-15, 17-24 and 27-33 are patentable 
over Jain. Appellants believe that pending Claims 5, 6, 16, 25 and 26 are patentable over 
Jain in view of Sullivan. As such, Appellants submit that Claims 1-33 are patentable over the 
cited references. 

Appellants respectfully request that the rejection of Claims 1-33 be reversed. The 
Appellants wish to encourage the Examiner or a member of the Board of Patent Appeals to 
telephone the Appellants' undersigned representative if it is felt that a telephone conference 
could expedite prosecution. 

Respectfully submitted, 
Wagner Blecher LLP 

Dated: 3/24/2008 /John P. Wagner, Jr./ 

John P. Wagner, Jr. 
Registration No.: 35,398 

Wagner Blecher LLP 
Westridge Business Park 
123 Westridge Drive 
Watsonville, CA 95076 

Phone: (408) 377-0500 
Facsimile: (831) 722-2350 
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VIII. Appendix - Clean Copy of Claims on Appeal 



1 . A method for culling view dependent visual data streams for a virtual 
environment, comprising: 

determining a view volume of a viewing participant within said virtual environment, 
wherein said view volume defines a field-of-view of said viewing participant within said 
virtual environment; 

determining a proximity of a representation of an observed object in said virtual 
environment to said view volume; and 

processing a view dependent visual data stream of said observed object only when 
said representation is within a specified proximity to said view volume. 

2. The method of Claim 1, further comprising: 

computing a three-dimensional model of said observed object, said three-dimensional 
model based on a plurality of real-time video streams taken of said observed object from a 
plurality of sample viewpoints. 

3. The method of Claim 2, further comprising: 

generating a view dependent video image stream by applying a new view synthesis 
technique to said three-dimensional model of said observed object, wherein said video image 
stream is generated from a viewpoint of said viewing participant. 

4. The method of Claim 1, further comprising: 
sending said visual data stream to said viewing participant. 
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5. The method of Claim 1, wherein said determining a view volume further 
comprises: 

determining a view direction of said viewing participant to define said view volume, 
wherein said view volume comprises a series of expanding cross-sections of a geometric 
object along said view direction from said viewing participant within said virtual 
environment. 

6. The method of Claim 5, wherein said geometric object comprises a four-sided 
rectangular plane. 

7. The method of Claim 1, wherein said determining a proximity of a representation 
of an observed object in said virtual environment to said view volume, further comprises: 

determining that said representation is within said specified proximity; 
determining when said representation is occluded in said view volume such that said 
observed is not visible to said viewing participant; and 

not generating said video image stream when said representation is occluded. 

8. The method of Claim 1, further comprising: 

providing for hysteresis and anticipation in delivering said video image stream to said 
viewing participant by defining an extended bounding volume that surrounds said observed 
object within said three-dimensional virtual environment, wherein said representation 
comprises said extended bounding volume when determining said proximity. 

9. The method of Claim 1, further comprising: 
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enabling a change in a location of said viewing participant within said 
three-dimensional virtual environment by determining a new view volume of said viewing 
participant within said virtual environment; 

determining when said representation falls within said new view volume; and 
generating a video image stream of said observed object from said three-dimensional 
model when said representation is within said specified proximity to said new view volume 

10. The method of Claim 1, further comprising: 

enabling a change in location of said observed object within said three-dimensional 
virtual environment and reflecting said change in location in said representation. 

1 1 . The method of Claim 1 , wherein said observed object comprises a local 
participant. 

12. The method of Claim 1, wherein said virtual environment comprises a three 
dimensional N-way virtual collaborative environment. 

13. A system for culling view dependent visual data for a virtual environment, 
comprising: 

a view volume generator for determining a view volume of a viewing participant 
within said virtual environment, wherein said view volume defines a field-of-view of said 
viewing participant within said virtual environment; 

a comparator for determining a proximity of a representation of an observed object in 
said virtual environment to said view volume; and 
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a processor for processing a view dependent visual data stream of said observed 
object only when said representation is within a specified proximity to said view volume. 

14. The system of Claim 13, wherein a source comprises: 

a model generator computing a three-dimensional model of said observed object that 
is based on a plurality of real-time video streams taken of said observed object from a 
plurality of sample viewpoints; and 

a new view synthesis module for generating a view dependent video image stream by 
applying a new view synthesis technique to said three-dimensional model of said observed 
object, wherein said video image stream is generated from a viewpoint of said viewing 
participant. 

15. The system of Claim 13, further comprising: 

a transmitter for sending said visual data stream to said viewing participant. 

16. The system of Claim 13, wherein said view volume generator determines a view 
direction of said viewing participant to define said view volume, wherein said view volume 
comprises a series of expanding cross-sections of a geometric object along said view 
direction from said viewing participant within said virtual environment 

17. The system of Claim 13, wherein said comparator determines when said 
representation is occluded in said view volume such that said viewing participant is unable to 
view said observed object, such that said video image stream is not generated when said 
representation is occluded. 
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18. The system of Claim 13, wherein said representation comprises an extended 
bounding volume that surrounds said observed object within said virtual environment, 
wherein said representation comprises said extended bounding volume when determining 
said proximity. 

19. The system of Claim 13, wherein said view volume generator enables a change 
in a location of said viewing participant to a new location within said virtual environment by 
changing said view volume of said viewing participant within said virtual environment to 
reflect said new location. 

20. The system of Claim 13, wherein said comparator enables a change in location 
of said observed object to a new location within said three-dimensional virtual environment 
and reflects said change in location in said representation. 

21. A computer system comprising: 
a processor; and 

a computer readable memory coupled to said processor and containing program 
instructions that, when executed, implement a method for culling view dependent visual data 
streams for a virtual environment, comprising: 

determining a view volume of a viewing participant within said virtual environment, 
wherein said view volume defines a field-of-view of said viewing participant within said 
virtual environment; 

determining a proximity of a representation of an observed object in said virtual 
environment to said view volume; and 
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processing a view dependent visual data stream of said observed object only when 
said representation is within a specified proximity to said view volume. 

22. The computer system of Claim 21, wherein said method further comprises: 
computing a three-dimensional model of said observed object, said three-dimensional 

model based on a plurality of real-time video streams taken of said observed object from a 
plurality of sample viewpoints. 

23. The computer system of Claim 22, wherein said method further comprises: 
generating a view dependent video image stream by applying a new view synthesis 

technique to said three-dimensional model of said observed object, wherein said video image 
stream is generated from a viewpoint of said viewing participant. 

24. The computer system of Claim 21, wherein said method further comprises: 
sending said visual data stream to said viewing participant. 

25. The computer system of Claim 21, wherein said determining a view volume in 
said method further comprises: 

determining a view direction of said viewing participant to define said view volume, 
wherein said view volume comprises a series of expanding cross-sections of a geometric 
object along said view direction from said viewing participant within said virtual 
environment. 

26. The computer system of Claim 25, wherein said geometric object comprises a 
four-sided rectangular plane. 
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27. The computer system of Claim 21, wherein said determining a proximity of a 
representation of an observed object in said virtual environment to said view volume in said 
method, further comprises: 

determining that said representation is within said specified proximity; 
determining when said representation is occluded in said view volume such that said 
observed is not visible to said viewing participant; and 

not generating said video image stream when said representation is occluded. 

28. The computer system of Claim 21, wherein said method further comprises: 
providing for hysteresis and anticipation in delivering said video image stream to said 

viewing participant by defining an extended bounding volume that surrounds said observed 
object within said three-dimensional virtual environment, wherein said representation 
comprises said extended bounding volume when determining said proximity. 

29. The computer system of Claim 21, wherein said method further comprises: 
enabling a change in a location of said viewing participant within said 

three-dimensional virtual environment by determining a new view volume of said viewing 

participant within said virtual environment; 

determining when said representation falls within said new view volume; and 
generating a video image stream of said observed object from said three-dimensional 

model when said representation is within said specified proximity to said new view volume 

30. The computer system of Claim 21 , wherein said method further comprises: 
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enabling a change in location of said observed object within said three-dimensional 
virtual environment and reflecting said change in location in said representation. 

3 1 . The computer system of Claim 21 , wherein said observed object comprises a 
local participant. 

32. The computer system of Claim 21 , wherein said virtual environment comprises a 
three dimensional N-way virtual collaborative environment. 

33. A computer readable medium containing executable instructions which, when 
executed in a processing system, causes the system to perform the steps for a method of 
culling view dependent visual data streams for a virtual environment, comprising: 

determining a view volume of a viewing participant within said virtual environment, 
wherein said view volume defines a field-of-view of said viewing participant within said 
virtual environment; 

determining a proximity of a representation of an observed object in said virtual 
environment to said view volume; and 

processing a view dependent visual data stream of said observed object only when 
said representation is within a specified proximity to said view volume. 
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IX. Evidence Appendix 
No evidence is herein appended. 
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X. Related Proceedings Appendix 
No related proceedings. 
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